Capturing incomplete information in resource allocation problems through numerical patterns

We look at the problem of optimizing complex operations with incomplete information where the missing information is revealed indirectly and imperfectly through historical decisions. Incomplete information is characterized by missing data elements governing operational behavior and unknown cost parameters. We assume some of this information may be indirectly captured in historical databases through flows characterizing resource movements. We can use these flows or other quantities derived from these flows as “numerical patterns” in our optimization model to reflect some of the incomplete information. Since regression trees represent probabilistic distributions of a numerical value specific to a vector of data elements known as a “state”, they drive our methodology for representing information in resource allocation models. We use a popular goodness-of-fit measure known as the Cramer-Von Mises metric based on the empirical distribution function (EDF) as the foundation of our approach. We then use a hybrid approach of solving a cost model with a term known as the “pattern metric” that minimizes the deviations of model decisions from observed quantities in a historical database. We see that when using this methodology, our problem of representing information pertaining to numerical patterns is combinatorial in the order statistics of the decision variables in the optimization model. We present a novel iterative method to solve this problem to get a stationary solution. Results with real-world data from a large freight railroad are presented.